Strict Minimum Message Length (SMML) is a statistical inference method widely cited (but only with informal arguments) as providing estimations that are consistent for general estimation problems. It is, however, almost invariably intractable to compute, for which reason only approximations of it (known as MML algorithms) are ever used in practice. We investigate the Neyman-Scott estimation problem, an oft-cited showcase for the consistency of MML, and show that even with a natural choice of prior, neither SMML nor its popular approximations are consistent for it, thereby providing a counterexample to the general claim. This is the first known explicit construction of an SMML solution for a natural, high-dimensional problem. We use the same novel construction methods to refute other claims regarding MML also appearing in the literature.

The minimum message length(MML) principle[7] is an information theoretic criterion that links data compression with statistical inference [6]. It has a number of useful properties and it has close connections with Kolmogorov complexity [8]. Using the MML principle to construct estimators is known to be NPhard in general [4] so it is common to use approximations in practice [6]. The term'strict minimum message length' (SMML) is used for the exact MML criterion, to distinguish it from the various approximations. The only known algorithm for calculating an SMML estimator is Farr's algorithm [4] which applies to data taking values in a finite set which is (in some sense) 1-dimensional. A method for calculating SMML estimators for 1-dimensional exponential families with continuous sufficient statistics was also recently given in [3]. However, calculating SMML estimators for higher-dimensional data is still a difficult problem.

The minimum message length(MML) principle[4] is an information theoretic criterion that links data compression with statistical inference [3]. It has a number of useful properties and it has close connections with Kolmogorov complexity [5]. Using the MML principle to construct estimators is known to be NPhard in general [1] so it is common to use approximations in practice [3]. The term'strict minimum message length' (SMML) is used to distinguish the exact MML criterion from these approximations. The only known algorithm for calculating an SMML estimator is Farr's algorithm [1] which applies to data taking values in a finite set which is (in some sense) 1-dimensional.